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If information science is to be considered a 
»» science" in the true sense of the vord, a set of general concepts 
and analytical expressions must be developed. Fundamental to this 
development is a rigorous and guantifiable measure of information. In 
previous papers a general framework, called a generalized information 
system, is suggest {$d which permits the development of these concepts 
and expressions. Through the use of this generalized model, ae have 
been able to define information quantitatively and in a rigorous 
manner. The formulation depends on the definition that "information 
is data of value in decision making** and leads to quantitative 
relationships between information and the value of a decision state. 
The value of the decision state is defined as the summation of the 
expected values of all the possible courses of action weighted by the 
probability of each course of action. A new measure for the 
information contained in a particular decision state is developed. 
The information is defined in terms of a two-choice deterministic 
situation which we call a "binary choice unit." This measure is 
universally applicable for all information that is concerned with the 
effectiveness of the data on the recipient. (Author) 
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^stract BEST COPY flVAilABLE 

1^ infona«it ion science 1$ to be considered 
A ^'science" in che cruti stsu^ of che word* a 
s^t of getier^I ccMicepts and anaLycical expres- 
sions mast be developed. Fundamantal CO this 
developxBcnt is a rigorous and quantifiable 
seasure of information, in previous papers a 
general franework* called a generalized inCor- 
fBAtion system « is suggested whi^h ptirmics the 
development of these concepts and expressions, 
through &he use of this generalized model, ve 
liave been able to define information quantita- 
tively and Lc. a rigorous nanner. 

The formulation depends cn the definition 
that itt/aj*T<i5ij?»: is Sxi'a jf ':atue .in dseision~ 
tmking and leads to quantitative relationships 
tietveen Informs .ion and the value of a decision 
state. The value of che decision state Is 
defined as the summation of the expected values 
of all the possible courses of action weighted 
by the probability of each course of action. 
A new measure for the infomatlon contained in 
a particular decision state l& developed: 

m 

i^l ^ 

«^ere n is the number of courses of action 
available to a decision-maker and rfa.J is the 
probability of each course of action." The 
iaformatlon is dafined in t£»nas of a two- 
choice deterministic situation which we call a 
*Sinary choice unit." This measure is univer- 
sally applicable for all information that is 
concerned with the effectiveness of the data 
upon the recipient* 

INTRODUCTIO!? 

The authors have been concerned with the 
development of a theory and franevork for 
establishing a ccnceplu.ti basis far informa- 
tion science. :iuch of this vork i3 aumaarisad 
In references (0, (9)» (10), and (11). 

If information science is to be consi- 
dered a "science" In the true sense of the 
word, a set of general concepts and analytical 
expressions regarding che flow of information 
must be do*/e loped. Fundamental to this devel- 
opment is a rigorous and qaintit::itivG measure 
of infomation. In this paper, such a measure 
is suggested and examined in the framework of 
a very general decision model. This formula- 
tion depends on the definition that infomation 



is data of in deeiavm^aHng and lea<£H" 

t9 the. establishment of quantitative relation' 
ships betwen information and che value of a 
decision state. 

A decision state is defined as a^represer. 
tation of the entire decisiot situation, it 
gives a coi^lete description of all of the el< 
ments which enter into a particular decision- 
making situation. It is shown in this paper 
cnat the value of the decision state may be 
defined as che summation of the expected valu. 
of all the possible courses of action wt^li^hte 
by the probability of each course of action, 
la synholic terms, 

m 

V(T>$) » Z P(aJE7CaJ. 
' M ^ ^ 

A oeasure of the amount of information 
contained in a particular decision state is 
si^ested as follows: 
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t^ere m is the number of possible cotxrsMS ol 
action available to the decision -maf:er and 
P(aJ is the probability of each course of 

action. The distribution of the ?(z.}*s 

reflects the decision-maker ♦s overall IncUm. 
tion towards each of the courses of action. 
Information is that quantity which changes this 
probability distribution. 

The information measure we surpcsc Is 
defined in terms of a two-choice deteministi 
situation in which the amount of infomation 
the decision state is one. Information i-s 
measured in terns of thcsa binary choice unit 
or t.c.u.*s. The measure is universally i??-: 
cable for all lnfonr.aticu chit is rcn-o— 
vlth the effectiveness of frc; n.osisa^: • -"^ 
recipient. It Is accordingly called nr.i : tio^x 
information and Is similar to that of '..•o.r.vt 
level three (6). Prooertios of this and 
related measures are also discussed. 

BACKGROUND 

Tlie term "information" h..^ n y*" 
of meanings. Shannon and Weaver (C ) c 
about Information in the technical s.r-^^-. 
That is, they are concerned vlth the ^ 
efficient way to transmit raessaRcs. <.;..iri«-> 
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tfonid (5> points out that it is sow generally 
* realized that "infona^ion theory" is not a 
rl'Vpl tp» or a substitute for, a general theory 
of sigQs. Shannon «ind Weaver** concera ia with 
the transnission of any message independent of 
its content. Y, Bar-HUlel (2) and D. H. 
I^cillay (A) clarify this situation. MacKay 
regards information as that which changes our 
representations, i.e., our signs. Gaining 
information Is cnus changing our expectations, 
i*e.a our dispositions to respc»id, caused by a 
sign. He distinguishes between selective and 
eemantic Information. Selective Information 
.'gives the information necessary to select the 
nessage itself and is not concerned with the 
eoatent of the message. Semantic information 
is concerned with the content of the message. 
SiasoKxi thus deals with selective infiommtion 
problems. Camap and Bar-Hillel (3) and 
fUnograd <8) are perhaps best known for their 
wrk in the area of semaatic information. 

the aforementioned views of infors^tion 
are two of the three approaches or levels 
identified by Weaver. The third level Is 
knom as the behavioral or effectiveness level 
and deals with the effect that information has 
ox the person using it. Ackoff (1) has dealt 
with information problems at this behavioral 
level. Our work lies in this area. 

DECISION-MAKING 

Charles Morris (5) has Identified three 
general requirements of action involved in the 
decision-making process. A dec is ion -maker must 
obtain information about the situati(»i in which 
he is to act, he must select among courses of 
action, md he must execute this alternative by 
some specific course of behavior. To effect a 
meaningful analysis of information, one must 
examine In detail that which makes decision- 
making such a challenging activity — 
uncertainty. After careful examination of 
decision models described in the literature, it 
became evident that these existing models do 
not provide a coioprehensive representation of 
the uncertainty that exists in decision-making. 
Host of the models are concerned solely with 
deeision-oiakers who have an advanced state of 
knowledge about the decision situation in 
qtKiStion. The information science aspects of 
decision theory must, however, cover compre- 
hensively not only those decision -makers who 
are expert but also those decision--iQakers who 
are average or rather poor. It is extremely 
Important in developing a formal role for 
Information science that alt levels of effec- 
tiveness of decision-makers be considered. 
For this purpose, a very general decision model 

j-uposed. 
ERIC 



A decision model consists of a ntanber of 
decision elements, including a set of courses 
of actiem, a set of possible outcomes* a goal 
or set of goals, a function relating outcomes 
to goal attainment, and a set of states of 
nature. 

The decision-maker usually views a com- 
plex decisicm situation in terms of courses of 
action and possible outcomes. Hu may be 
uncertain about what outcomes will occur when 



a particular course of action is executed. 
This uncertainty associated with the executi«>. 
of the alternatives is what we call exeoutir.^^ 
uncertainty* K second type of uncertainty 
identified is goal unoeftainty. The decision 
maker may have only a vague notion of the go^^ 
to i^ch he aspires, and he may also be uncer- 
tain as to the degree to which each of the ou* 
comes will satisfy the various goals* The 
third type of uncertainty tdilch the decision- 
maker confrosits is that concerned with the 
states of nature. He may not be able to idea- 
tify all the possible states, but even if he 
could, he may still be uncertain as to the 
relationship between the set of states and th« 
other decisicm elements. This is termed 
environmental moeptcdnty , A complete model <r 
a coiq>lex decision situation must deal explic- 
itly with all of these types of uncertainty. 
The ccmceptual decision model suggested expll 
itly recognizes all of the decision elements 
as well as the associated sources of 
uncertainty. 



MATHEMAtlCAt gEPRRSESTATTO?! 
OP WE DECISION MOBEL 

A decision 'laaker makes a Sequence of 
related choices from aiaong a discrete set of 
alternatives A * {a„ a., a^}. The 

aunber of elements in this set may not be 
coastant over time since there may be uncer*^ 
talnty vith regard to these elements* The 
execution of a particular course of action 
results in the occurrence of one of a set of 
possible outcoiaes 0 « {^7^^ 

The number n may also vary over time 
tlonal uncertainty, i.e.^ uncertainty as to t: 
relationship between a particular course of 
action a* and an outcome a» will be denoted b 

the subjective probability estimate 'J.j, the 

likelihood that the execution of course of 

action a* will result in outcome The set 

^ ■ J 

of relevant states of nature will be denoted 
by S ^ {s^, •••^ Sr^^ ••.ji $ } where r may al^- 



Sxecu- 



vary over time according to the decision- 
maker's current understanding of the decision 
situation^ The probabilities of occurrence i> 
the states will be denoted by the Hubjaccivi^' 

estimates PCaJp .♦•^ Pf&^Jp '^^^ 
J t< 1* 

values assigned to the decision outcomes that 

reflect the relative value of each outcome 

with respect to goal attaiiuaent will be 

denoted by iv(Oj)}. 

The decision elements 0, and ivC^^*)^ 

.... 

are dependent upon the state of the exti»ni.il 
environment. For example ♦ courses of actlcn 
v^ich seem reasonable under one sec of contii" 
tions may be wrong under other circumstance;** 
These dependencies can be incorporated in t'u 
model by defining the sets A and - to rv'fl •* 
the decision-maker's current understandin ♦ 
the courses of action and the cut comas lor 
each of the states of nature. Also, a st.a ^-'^ 

{iJ./} and i'J.Jj J) can be identified for 

each state of n.i re. Figure 1 dcpictj^ tni 
suggested mathetn.-t' ical representation f^^r 



particular state of nature Sj^* 
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Figure 1. The decision matrix for 
Che kth state of nature 
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Hsm details can be found In (7) • 

the oathemat leal decl$icr> model consists 
of this decision matrix cosecher with some 
deeision criteria vhich, when applied to the 
decis$<»t matrix, results in the selection of 
pfohabllities of courses of action. The actual 
decisioit rule that is used is dependent on the 
decisioa-aaker's own attitude toward uncer- 
taiiity* For example, if the decision-maker is 
omservative, he may select that course of 
iaiictiixi which maximizes his minimum possible 
gain. There are many different criteria which 
coa .jbe used, but in this analysis we will 
assume that the decision-maker assigns proba- 
bilities to the alternatives which are propor- 
tional to their relative expected value. The 
decision rule that we have raconmended is a 
wtf reasonable one, and a number of Interest- 
ing results follow from this rule. The 
expected value, is defined by 

• r ■ ■ n , ^ 
SVCaJ'- Z PCs:,) I t^jj v. (a,)* (1) 

That iSf the expected value of each alternative 
is the sum of all the possible values weighted 
by their probabilities of occurrence. 

This generalized decision model provides 
a framework for a formal and comprehensive 
representation of uncertainty in decision- 
waking. As such, it also provides a suitable 
framework for examining the role of informa- 
tion in decision-making tliat is also formal 
and comprehensive* 



The effect of information is to change th& 
deci8ioit*sia!»r' s representation of the various 
types of uncertainty. His decision model at 
tine t 4' 1 will be a revised version of his 
model at time t. The way in which a particuU 
decision-maker utilizes information to roviso 
his representation of the various types of 
uncertainty is highly individualistic. The 
g^eralized decision model perrdts the appli- 
cation of a large ntunber of possible learning 
rules. 

A QUANTITATIVE MEASURE OF lyyOSHATTO!! 

Although the separate effects of the 
various types of uncertainty are clearly 
important, they are only of significance in 
their combined effect on the decision-maker* a 
understanding of the situation* The amount 
value of the information contained in a data 
set can be meaoingfully expressed only in t err 
of the total effect of the data on the 
decision-maker's model of the decision 
situation. 

Regardless of what decision rule a 
decision-maker is utilizing, it is possible tv 
obtain a distribution that reflects the 
decision-maker's overall inclination toward 
the various courses of action. We asstane, as 
already suggested, that the decision-maker 
chooses a course of action with a probabiH*-^' 
proportional to its relative expected valuf. 
Thus, P(aJ is defined by 



Pfa.) 



m 

Z SVCaJ 



where EVCcZj) Is given by equacion (1>. 

! ■■ ■ ■ ■ 

The decision matrix representation shown 
'In Figure 1 defines completely the entire 
Idedsion situation. It explicitly relates 
«<mr8es of action tor (1) observable outcomesi 

values of these various :»utcoiaes to the 
deelsion-reaker; and (3) the states of nature. 
pti9 matrix may thus be ci^fimsd to be the 
'^aiaian state of the dsaiaion'-^aksT, This, 
decision state is a complete description o€ all 
9f the el^nenta which enter into any decision- 
O^lng situation. The uncertainty in this 
decision state can be measured and calculated, 
and this will be Indicated later. The impact 
of information on reducing the uncertainty in 
this state function aiay then serve as a measure 
of information. Since the information can also 
be related directly to the values of the various 
outcomes » it is therefore also possible to cal- 
culate directly the value as well as the mount 
of the information. 

Equation (1) provides the relationship 
which yields the Expected Value (EV) of any 
course of action a.. The uncertainty which 

exists for any decision stace vill be a func^ 
tlon of the mean square variance of the expec- 
ted values of the various courses of action* 
For example^ if all the EV^3 are the same» the 
decision-maker will totally uncertain as to 
which alternative to choost-^ and the variance 
will be zeroe If the decision •maker is com-- 
pletely certain as to his course of action* 
then all of the £^7'^ will be aero but one ^sn^ich 
will be finite. For such a situation* the 
variance can be shown to be a maKimunie 

the mean square vari^ce* o^* is defined 

(3) 



and the taean* y, is defined as 



[r EV(aAA, 



(4) 



where m is the number of possible courses of 
action. 



can now define the valu^ of the 
^aiaion etate as the suafflation of the expected 
values of all the possible courses of action 
vi»t?5hted by the probability of each course of 
action. In symbolic terms. 



V(DS} 



m 

Z 



p(aJ E7(aJ. 
% t 



(5> 



The information contained in a decision 
^•tato is related to the lacai. square variance of 
t^v expected values of the courBos of action. 
r«» be precise, the information Is related net 
»it biit rather to g-^/u*', Jiinc? information 
'Utniid be »fasured by relating it to the 



jirariance measured in units of the mean. Thi. 
must be the case since a given variance of .y 
will clearly be much less significant when tu 
mean of the EV*9 is large than <4ien the me^.^ 
small* The quantity o^/u^, from equations ( 
fflxd (5), is 



i 



It is perhaps more meaningful to view this 
relationship in terms of the PCaJ's. With t 

use of equations (2), (4), and CS>, one obtai 



m Z {PCaJ}^ - 1, 



( 



This quantity possesses the desired 
properties for an information measure. The 
more uncertain the decision-maker is, the les 
the aoount of information in his decision 
state. Thus, we define this fundamental 
quantity to be the amount of information in c 
particular decision state. That is, 

• m 

Note that this quantity has a minijaum of zerr 

when all the PCa_^) 'a are equal to ~. This is 

complete uncertainty. The quantity has a jaa>: 
mum of m - 1 in the case of complete certain: 
where one of the P(aJ 'a is one and the other 
are zero. 
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. liben thete are only two possible coursQ^ 
* of diitiosiy T will Asaune values frora 2era to 
otie« It will be equal to t)ne undtir conditic. 
of certainty . t ♦ . ♦ wlu'n t iic p rab.th i 1 i ty v.^ f 
choosing one course of" action la one and the 
other probability is zero. Accordingly, ve 
will define the unit of information in terms 
a deterainlstlc two-choice situation. This 
unit is called a binary choice unit, or b.c. 

When there art» "t possible courses of 
action, then the rajxlnum anount of inforaati 
from Equation (S> is seen to be n - J b.c.u.' 
This is in agreement with a well-known prin* 
pie that a minimui^i of - I choices fron pai: 
of alternatives is required n^hen there are 
alternatives to consider. This is pointed vj* 
in (1). More escplicitly, if •?? - 2 choices ^ 
required and the laaxiraum amount of inforr.at: 
in each choice is one, then the maxiraur. 
of information is '•7 - 2. Analogously the n." 
mum amount of information is zero* 

In summary. Equations (6> and (S) pr.A i 
us with a method of obtaining the value of 
information in addition to the amount. 

Ve can now define a quantity called thf 
index of determinism. 

Sote that this is just T/'-^, this qu.intitv 
assumes the value zero when 4U the 



f(aJ*9 are equal (the case of total uncer- 
taintf) and the value I - z* vhen the situation 

"7 

is eonpletely deterministic. For large m, it 
will approach unity. Thus the index of 
detevninian is a quantity -jar^yinj from ssr*o to 
one irfiich measures the deteminim of the 
dsffisim state, 

\ • PROPERTIES OF THE INFO!tttATION MEASURE 

\ The suggested information measure 
l^ssesses a number of desirable properties. It 
is defined in terms of a fundamental unit of 
■easure which we tensed the binary choice unit, 
or b.c*tt.« 

* 

The consideration of additional butr 
lilghly unlikely courses of action has a very 
small effect on the amount of information in 
the decision state. 

ERIC 



Aiu>ther desirable property of the 
infenBacioo measure is "sequential additivity.** 
The amotmt of information in a decision state 
can be measured all at once or the process can 
be broken up into several steps with the con- 
sideration of a few alternatives at a time, 
iegardless of which method is used, the aaount 
of information in the entire decision state is 
the sane* 

A measure of the amount of information in 
m data set or message can be arrived at by 
C4^uting the difference in the amotmt of 

#nBation in the decision state before and 
<.. .er receipt of the data. That is, the amount 
of Intoxmtion is arrived at by considering the 
intact this new data has on the decision- 
tucker's decision state. In symbolic terms, 

the amount of information in data set D, 

is 

KDi • 1^^^ . , (10) 

«l»re J^j^^jj **** ^-t amounts of Informa- 

tion in the decision state after and before 
receipt of the data set. 

It should be noted that the amount of 
information in a data set may be either posi- 
tive or negative. In general, positive infor- 
maticm sharpens or retinas the decision-maker's 
understanding of the situation in that it 
either reduces the number of structural compo- 
nents in the model or reduces the -dispersion 
In one or more of the various probability 
distributions in the model. Negative informa- 
tion, on the other hand, either Increases the 
number of structural components (e.g., the 
addition to the model of a previously unknown 
alternative or outcome) or increases the dis- 
persion in the various distributions. Negative 
information, despite a possible connotation of 
the term, does represent information that is 
of significance to the decision-maker. 

DISCUSSION AND SUMMVRY 

formal measure for the .mount and the 
value of Information contained in a data set 
or message has been sunj'ested. It quantifies 
information in terms of Its effect on the 



state of the decislon-taaker, where a (Jecisto. 
state is defined so that It roprefkJnts a c. 
ple^ description the decislon-maker^s 
overell level of understanding about a partl^ 
Cttlar decision situation at a particular poi; 
ia ttwe. This measure Is universally appU* 
cafcle for pragmatic information. This is 
equivalent to Weaver's level three which is 
concerned vith the effects of the nessage upt 
the recipient* 

la orAtr to evaluate this measure of 
iafotmation« it is convenient to use a gener. 
Ized information svstem model. The use of 
this ooiftel then permits the evaluation of thi 
fl^asure of information in terms of the reduc- 
^i-OR ol uacettsdnty. This evaluation could • 
inaMie iii terms of any kind of a decision rule. 
We have suggested a reasonable decision rule 
that can be used, and we have developed 
relationships based on this rule. Virtually 
mf other decisicn rule could be used for 
evaluating the effects of the various uncer- 
tainties referred to. It would also be 
possible to evaluate the decision state of t: 
decision-maker iii a purely descriptive sense. 

In summary, the proposed information 
measure is a function of the effect chat a s« 
of data has on a decision-maker's decision 
state. .This decision state is defined in su 
a way that it reflects the decision-maker's 
understanding of a particular decision situa- 
tion at a particular point in tine. Hence, . 
Is a situation dependent and tine dependent 
measure. Clearly it must be time and situa- 
tion dependent since the same data will have 
different significance to different decision- 
makers at any point in time or to the sana 
decision-maker at different times. 

FCTURE PLAN'S 

The information measure suggested in tht 
previous sections leads to a measure of the 
pragmatic information content of a data set 
for a pccPticulcu* decision -maker at a vxi^Z' 
point in time. The data acquired, processed, 
stored, and disseminated by an information 
system may be used, however, as a resource b 
i:2riou3 decision-makers at '*2X*i:-'t3 points in 
time. Hence, In the design and development 
information systems, there exists a problen 
it^iise level of complexity is an order of ma:: 
tude above that of the primary problems ad- 
dressed in this study —the problen of quan 
fying the information contained in a data 
In terms of its overall usefulness for a 



One possible approach to this prt^bto «'f 
assigning .1 numbi^r to a data s<»t to InJior. • 
its compos Ite information content wo&ld be tQ 
start by detemining ttm relationship bctt^«?t^^ 
tt^ effectiveness of a decision-maker and the 
pragmatic in format i<Mi content of the data sci 
for this decision-maker. Since what is rc^i 
dsfiired is some indication of the informatli'* 
content of this data set for this declsii>ti- 
maker over a period of time, one may detenu * 

some index f(D) of the average infotfiiatlon.- 
contained in data set D over some period v>( 
time. Then, if one were to develop a mtNisjne 



the effectiveness of each of the decision- 
Bakers for whom this data set serves as a 
resource. It would be possible to formulate an 
infamation profiU for the data set. Such an 
info naat Ion profile would indicate the average 
ittfomiaticn content of a data set as a function 
of deelslon-aaker-effectiveness. 

If stt^ a profile could be detennined for 
every data set to be stored in an information 
fljstea, then soma number derived from this 
profile could serve as an index of the composite 
value of this data set. This method would be 
of ttajor importance for the development of a 
sound procedure for the design of more 
effective information systems. . . 

(1) Ackoff , R. L. "Towards a Behavioral 
Theory of Commiinicacion." Hanagement 
Science. (1958), 218-234. 

CZ) Bar-Hillel« V. **An Examination of 
Information Theory." Philosophy of 
Science . 22 (1955), 86-103. 

(3) Camap, R«, and Bar-Hillel, Y. An Outline 
of a Theory of Semantic Information. 
Technical Report No. 247, Research 
Laboratory of Electronics, M.I.T., 1952. 

(4) KacKay» D. M. "In Search of Basic 
Symbols" and "The Komenclature of 
Znfomation Theory,** in Cybernetics; 
Transactions of the Eighth Congress, 
Heins von Foerster, ed. (.\'ew York: Macy 
Foundation, 1952). 

(5) Morris, Charles, Signification and 
Significance . (Cambridge, Massachusetts: 
The M.I.T. Press, 1964). 



(6) Shannon, Claude, and Weaver, Warren . Tte 
Mathemattcal Theory ot Comraun teat ion * 
(Urbana, Illinois: Unlvaraicy Illinois 
Press, 1949). 

(7) Whittemore, Bruce J., and Yovits, M. C. 
"A Generalized Conceptual Development for 
the Analysis and Flow ol Information." 
Journal of the .Vnerlcaa Society for 
Information Science, *>!ay-June 1973, 

Vol. 24 (So. 3), 221-231. 

C8) tflnograd, T. "Understanding Natural 

Language." Co gnitive Psvcholosy, 3, 1972. 

(9) Yovits, M. C. "A Theoretical Franevork 

for the* Development of Information Science,** 
in Infonti ation Scienccy Its Scope. Objects 
of Research and Probiwnid , edited by 
A. t. Mikhailov, Publication of 
International Federation of Documentation 
Committee on Research on the Theoretical 
Basis of Information « 1974. 

(10) Yovits, M. C. **Infomatlon Sciences 
Toward the Development of a True Scientific 
Discipline.*' American Docutnontation, 20 
(No, 4)(1969). 3b9-7f^. 



(11) tovlts, M. C, and Ernst, R. L. 
**Generalized Information Systems: 
Consequences for Information Transfer," 
Itt People and Information , H. B. PcplnsV, 
•d. (New York: Petgomon Fress, 1969). 



